The following algorithms describe how to create a "virtual camera" viewing system in which you have a camera positioned anywhere in a 3D world pointing to an arbitrary focus point. Camera Z panning (roll) is also possible by defining a direction vector.

Equations

O = observer position (camera)
A = aimpoint position (focus)
R = direction point position (up)

N = normalize (Ax - Ox, Ay - Oy, Az - Oz)
D = normalize (Rx - Ox, Ry - Oy, Rz - Oz)
V = D - (D . N) * N
U = cross product (V,N)

			  | Ux Uy Uz |
View Matrix = | Vx Vy Vz |
			  | Nx Ny Nz |

Note that "." means dot product and "*" means multiplying each component of the N vector with the dot product. (scalar / vector multiplication)

Hints

• If you wish to specify the angle for camera Z panning yourself:

  a = angle for camera Z-panning (roll)
  D = [sin(a), cos(a), 0]

• You must take care of the special case O=A ; don't alter the view matrix and use the previous one. If O=A happens at the first drawing frame and you don't have a previous view matrix use this instead:

  				 | 1 0 0 |
     View Matrix = | 0 1 0 |
  				 | 0 0 1 |
  
  

• You must take care of the special case where vector N is parallel to vector D, ie. abs(D.N) = 1. I have solved this special case by rolling the components of D like this:

  D = (Dz, Dx, Dy)

• Another special case is when O=R ; there will be no D then. In this case either use previous D or create a fixed one:

  D = (0, 1, 0)

  This is, of course, not needed if you specify D yourself (see above).

• In theory, vectors U & V don't need normalizing but it is advised to normalize those if you are using fixed point math instead of floats in order to avoid overflows caused by precision errors.

I use the above algorithms in my 3D engine and they work perfectly.

Thanks must go to Silver Eagle/Pygmy Projects, Celebrandil/Phenomena and Scout/C-Lous for their (kind) support.